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The Science Process Inventoi y (SPI) is one of several 
instruments developed during the 1960 's to assess student knowledge 
about the nature and processes of science. SPI (form D) has a 
single scale comprised of 135 sta::ements to which students "agree" 
or "disagree". This instrument has been used in a number of 
research studies. 

A single test score does not provide educators and curriculum 
developers much specific information for modifying instruction. 
It was hoped that a number of independent subscales might be found 
among the 135 test items. This study used a random subsample of 
435 students who were included in the Harvard Project Physics 
summative evaluation which used a national random sample of physics 
classes in the United States o 



Because of the very low correlations between the items, factor 
analysis of the total test has not provided interpretable factors. 
Efforts to group the items into subscales based on the original 
conceptual organization of the test items also failed to produce 
reliable scales. However, a factor analysis of 43 items selected 
on the basis of moderate difficulty level and demonstrated discrim- 
inating power did suggest five factor scales of 3-4 items each. 
These "protoscales " hold promise for developing useful scales of 
10-20 similar items. 

Several issues related to the construction of a multidimensional 
instrument to measure student knowledge about the nature and 
processes of science are also discussed. These include: 1) the 
philosophical bias of the test, 2) criteria for levels of difficulty 
and discriminating power, 3) appropriate response format, 4) need 
for a theoretical model, 5) need for development of a series of 
instruments, and 6) criteria for assuring a multidimensional 
instrument will be used by practitioners as well as researchers. 
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Introduction 

The Science Process Inventory (SPI) is one of several 
instruments developed during the 1960 's to assess student 
knowledge about the nature and processes of science} Since 
"knowledge" is prerequisite to higher levels of the Bloom Taxon- 
omy, reliable measures of student knowledge in this area should 
provide valuable information to both teachers and curriculum 
developers. However, the single scale score for all of the 
135 items in SPI is a gross measure which provides little 
specific guid-^nce for modifying instruction. 

This study explored the possibility of using items from SPI 
to describe student knowledge about several dimensions of the 
nature and processes of science. In general, divisions such 
as "Classification" or "Theories" did not produce usable subscales; 
however, five interpretable "protoscales" each containing 3-5 
items with reliabilities of 0.4-0.5 were identified. These 
protoscales suggest that relevant dimensions will reflect differ- 
ent philosophies of science such as the "Realist" or "Instrument- 
alist"; beliefs such as "Nature-Understandable" or "Science-Tentative" 
and specific concepts such as "Measurement-Approximate". In 
addition to encouraging continued efforts to develop a multi- 
dimensional instrument/ this study also suggests several guidelines 
for future work. 
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Some Background Information on SPI 

The Science Process Inventory (form L) contains 135 state- 
ments to which students are asked to ''agree" or "disagree"/ e.g. 

25. A theory in science may be modified in D 
light of new evidence. 

These statements were drawn from several prominent books on the 

2 

history and philosophy of science. Only those concepts which 
occured in a majority of the books surveyed were included 

in the instrument. A copy of form D of the Science Process 
Inventory has been included in Appendix A. 

Norms for the 150 item form C were obtained using a sample 
of 1283 senior high school students (grades 10-12) in two 

3 

Wisconsin high schools. The results as reported by Welch 

are shown in Table 1. The differences between grades 11 and 12 

are significant (p < .05). 



TABLE 1 — Norm Data for SPI {form C) 



grade 


N 


rel . 


std. error 


mean 


std. dev. 


range 


12 


444 


.80 


4.6 


108.8 


10.4 


77-132 


11 


403 


.79 


4.8 


106.8 


10.5 


70-132 


10 


436 


.78 


4.8 


107.0 


10.4 


70-134 




1283 






107.5 


10.4 





Welch also used SPI to compare high school students, high 

4 

school teachers/ and a group of scientists (see Table 2). The 

significant differences (p^.05) among these groups was inter- 
id 
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as an indication of the validity of the instrument. 



TABLE 2 — Comparison of three groups on SPI 

g^o^P N mean std dev 

High School Students 1283 107.5 10.3 

High School Teachers 16 129,4 6.7 

Scientists 19 135.0 4.7 



A shorter form of SPI was used in the Harvard Project 
Physics (HPP) summative evaluation (1967-68) which included 
a national random sample of physics classes^ Statistics for 
a random subsample of approximately 1/4 of the students in the 
HPP study are reported in Table 3. The high mean 



TABLE 3 — Norm Data for SPI (form D) (HPP data) 



N 


rel . 


std. error 


mean 


std. dev. 


435 


.76 


4.0 


107„0 


8.14 



of this sample reflects the increased homogeneity of the universe 
of students who elect to take physics as compared to all students 
enrolled in high school. The physics students in the HPP sample 
had a mean IQ of 117 (Henmon-Nelson ) , and SPI has been shown 
to be correlated with IQ r^.6l3 A (2x3) multivariate an^ '.ysis 
of variance (course x IQ) showed a significant IQ effect (p<.01). 
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but no course or interaction effects between the HPP experimental 

g 

and the control classes. 

SPI has also been used in several other studies including 

9 

the evaluation of Physical Science for the Non-Scientist , and 
Aikenhead has used items from SPI to investigate alternate methods 

for assessing student knowledge about the nature and processes 

^ . 10 
of science. 

Some General Comments about SPI 

There are several general questions relating to tests 
which measure student knowledge about the nature and processes 
of science. Three areas of concern in this study include 
1) the philosophical bias of the instrument, 2) criteria for 
the level of difficulty and discriminating power of items, and 
3) the choice of paper and pencil response format. 

The first of these concerns is the tension which exists 
within SPI between two philosophies of science — the Realist 
and the Instrumentalist. A Realist assumes that an external 
REALITY actually exists and that the pursuit of science has 
helped mankind make successively closer approximations to TRUTH. 
This philosophical perspective was characteristic of the physical 
models of Newtonian Classical physics which, at the time, 
appeared to have resolved the major riddles of the universe. 
The "correct" response to several SPI items reflects the Realistic 
philosophy, e.g. 
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69. Those people who carry on the practice of A (D^ 

science assume that: matter is an idea, 
not reality. 

The development of atomic and electromagnetic theory in 
the late 19th and 20th Centuries brought the Realistic perspective 
into question. The physical models which were proposed to 
explain phenomena such as polarization and propagation of 
electromagnetic waves became increasingly absurd and were 
ultimately replaced by mathematical equations. No longer was 
it necessary to provide a physical model to explain how phenom- 
ena occurred — an internally consistant set of equations which 

accounted for the phenomena would suffice. Further, the work 
11 

of Thomas Kuhn demonstrated that the major scientific revolu- 
tions have resulted in fundamental restructuring of "Reality". 
The immobile earth became a fleeting planet. Phlogiston 

evaporated. Time and space have been combined and made relative. 

1 2 

This new perspective is reflected in the Instrumentalist 
philosophy which has abandoned the search for "Truth" and is 
content with a science which is internally consistant for the 
phenomena as scientists currently perceive them. A student 
who holds the sophisticated Instrumentalist philosophy will be 
penalized in responding to many of the items in SPI. 

One of the protoscales identified in this study also suggests 
that some students hold a naively "Literal" interpretation of 
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scientific models. For example, they tend to agree with 
statements such as 

35 o (rhe Bohr model of the atoml is a scaled-up A (d) 
picture of what scientists have seen in 
their microscopes . 

It is important that instruments intended to measure student 
knowledge about the nature and processes of science explicitly 
recognize and hopefully measure differences such as those among 
the scientific Literalist, Realist, and Instrumentalist. 

The second concern is that most of the items in SPI are 
very easy (see Table 4). The average item difficulty is approx- 

Insert Table 4 about here 

imately 0.8 with 45% of the items higher than 0o90. 

Figure 1 shows the standard deviation (cr) of the SPI items 
as a function of their difficulty. This is the binomial distri- 
bution (solid curve) for which the variance decreases very 

rapidly as the probability of the event approaches 0 or 1 . 
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If an instrument contains only very easy items/ the distribution 
of scores will be skewed to the right making it difficult to 
discriminate among the "better" students. Likewise, a test 
composed of very hard items fails to discriminate among the 
"less able" students. 

For a mastery test/ a high difficulty level is desirable 
because the purpose of the instrument is to demonstrate that 
students exceed some minimum level of competence. There is no 
need to discriminate among students who pass the test. However, 
discrimination is paramount to an instrument intended to describe 
differences among students. It is also desirable that the test 
accomplish its purpose in as parsimonious manner as possible. 
The norm data for form C of SPI (Table 1) show a range of 70-134. 
Nearly half of the items in the test provide little 
information about differences among students. 

For maximum discr imination, the mean item difficulty 
should be around 0.5. Whether the difficulty level of all 
items should be 0.5 or represent a range of difficulties with 
a mean near 0.5 depends upon the nature of tlie test. If the 
test is assumed to contain independent items which measure a 
variety of JonceptS/ then it is best to include items with 
uniform difficulty near 0.5 which provides maximum positi^ve 
and negative variation on each item. However, if the scale is 
intended to measure a single concept, then the items will be 
"intercorrelated and a range of difficulties should be included."^ 
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Some might argue against choosing a mean difficulty near 
0.5 because the "guessing" level for an " agree" -"disagree" 
response format is also 0.5. A chimpanzee marking responses 
at random would receive an "average score" on the test. However, 
contrary to the classroom behavior of some students, the data 
do not suggest that many students respond to the items in SPI 
as would their hypothetical primate counterpart. If guessing 
were a serious problem, then it would be reflected in very low 
discriminating power for the items with d.i f ficulty levels near 
0.5. If fact, many of the most discriminating items in the 
subscales are those with modest difficulty levels of 0.4 to 0.6 
(see Appendix B) . 

The fact that a large portion of the SPI items are very 

easy for high school students suggests that they already have 

an acceptable understanding of many concepts about the nature 

and processes of science. Perhaps many of these less difficult 

items could be included in a simpler instrument appropriate for 

14 

upper grade school and early junior high, ^ 

The question of guessing could also be made moot by a change 
in the response format. If a four point scale (AA A D DD) 
with a confidence scale (L M H) were used, guessing would be 

15 

minimized and students would have more latitude in responding. 

These concerns preclude generating a multidimensional instru- 
ment directly from the existing items in SPI; however, the following 
- ERs^C search for subscales was valuable in identifying possible 
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Approach to the Analysis 

Aikenhead factor analyzed SPI in total, but found 

1 6 

factors which could not be readily interpretea. However, 
his analysis did establish that SPI does not contain a 
strong general factor. 

The difficulty in factor analyzing the total test arises 
from the very low intercorrelations among the items. A portion 
of the correlation matrix has been reproduced in Table 5. 

Insert Table 5 about here 

17 

Cronback notes that for items found in psychometric tests, 
the correlation among items is ordinarily below 0.3. Although 
several of the items correlate 0.1 to 0.3, the relationships 
are too weak to allow meaningful factor analysis of the total 
135 items. 

Relatively low phi correlations (^^^j) expected 
because the maximum possible correlation between two dichotomous 
variables depends upon their relative difficulty and the homo- 
geneity of the items as indicated by their tetrachoric correlation 

1 8 

(r^ j_) (see Figure 2). If two homogeneous items 1.00) 
uet *ce*c 

have identical levels of difficulty (p_^ = P ^ ) * then they can be 
perfectly correlated. However, as their levels of difficulty 
diverge, the maximum possible correlation between them rapidly 
decreases . 
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As the items become less homogeneous, the effect of item 
difficulty becomes less pronounced, but the maximum correlation 
is severely suppressed « 




Relation of 0^/ to Pi and for Several liCvels of Correlation. 

Figure 2 



Given these constraints, two efforts were made to examine 
the SPI items for possible subscales. The first was to hypothesize 
a set of subscales based on Welch's classification of the items. 
This did not result in useful subscales. The second effort was 
to factor analyze a sample of items which met certain criteria 
of difficulty and discriminating power. This factor analysis 
produced five promising ''protoscales" . 
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The 13 Hypothesized Subscales 

An effort to increase homogeneity among items was made by 

grouping items which appeared to be related into 13 hypothesized 

subscales. These subscales are similar to the original organi- 

19 

zation suggested by Welch (see Table 6) with many of the 

Insert Table 6 about here 

subdivisions combined. Fourteen items were not included in any 
of the subscales. A list of the items and statistical data for 
each subscale is provided in Appendix B, A summary of these 
data is presented in Table 7. 

Insert Table 7 about here 

Given the homogeneity of the student sample and the reduced 
number of items in each subscale, we did not expect them to have 
particularly high reliabilities. Indeed, the reliabilities 
range from 0.12 to 0.50, too low for the scales to be useful 
or to significantly increase their reliability by simply 
adding more items o More sophisticated procedures are required. 

The first effort to im.prove the hypothesized subscales 
was to select only the "best" items. The resultant sub-subscales 
contained only 3-4 items, but it was hoped that they would 
provide a strong nucleus of items around which more could be 
written. If a scale of three items has a reliability of 0.4, 
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then a scale containing ten similar items can be expected to 
have a reliability approaching 0.7. The selection criteria 
for items was based on the following analysis. 

The internal reliability of an instrument can be found 
using the general formula: 



^tt 



n - 1 



Where n is the number of items in the test, is the variance 
of the test scores, and is the variance of item score after 
weighting. 

If students respond to the items of the test at random, 
then the total observed variance for the test will be the sum 
of the item variances, the quantity in the bracket becomes 
zero, and the reliability of the test will be zero. However, 
if individual students consistently score above or below the 
mean for the items of the test, then the observed variance of 
test scores will be greater than the sum of the item variances 
and the reliability of the test will be O'^r^^^l. 

Since the item variance (v^) is fixed for a given sample, an 
effort was made to increase the test variance (V^) by selecting 
items which were both discriminating and of moderate difficulty. 
The discriminating power was the point biserial correlation 
between the item and the subscale. A range of items with modest 
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levels of difficulty was desired because the subscales were 
assumed to be homogeneous. The following criteria were used 
to select items: 

1) r^^>0..4 

2) 0.2^ Diff ^ 0.8 

Six of the subscales contained three or more items which met 
these criteria. They have been reported in Table 8. 

Insert Table 8 about here 

This procedure resulted in groups of highly discriminating 
items; however, the reliabilities for all except two of the 
sub-subscales are disasterously low. If subscales do exist, 
they are not the obvious catagories such as "Laws", "Hypotheses", 
etc . 

Factor Analysis of Selected SPI Items 

Thus far we have shown that if dimensions of knowing about 
Lhe nature and processes of science exist, they are considerably 
more subtle than might have been expected* We have also 

identified a pool of 43 items which have modest difficulty and 
demonstrated discrimination. While these items need not repre- 
sent all of the "useful" items in SPI, they do provide a reasonable 
sample for using factor anal^ 2 techniques to discover possible 
relations. 

O 
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TABLE 8 — Selected Items from Some Subscales 



1 Subscale 


Dif 


Dis 


# 


Mean 




KR#20 


l-{2) 


Universe: Orderly & Understandable 






4 


2.32 


1.18 


0.464 


3 {120- 74) MIND UNDRST NTUR 

4 (120- 75) NATUR NVR UNDRSTD 

5 (120- 77) PBMS 2 CMPLX 2 EX 
1 8 (130-119) EXPMTS CONSISTANT 


.75 
.43 
.66 
.49 

- - 


.59 
.72 
.65 
.53 


• - 




^ . — - — ■ 


. 




Causality 






3 


2.22 


0.77 


0. 067 


i 
I 

i 

1 


4 (140- 68) AB SAME TIME — CAUS 

5 {140- 70) EVNTS HV DISC CAUS 

6 (140- 71) DA-DB~A CAUSES B 


.80 
.73 
.69 


.51 
.68 
.58 


i 

1 

i 

i 


I 

f 


j 

: 




7-(2) 


Experimentation 






3 


1.79 


0.85 


-.001 




3 (240- 30) EXPMT PRV LWS NAT 

6 (240- 43) CONTROL IN EXPMT 

8 (24 0- 57) EXPMT ALLOW CONTROL 

1 ■ 


.53 
.65 
.61 


.58 
.57 
i .59 


! 

1 - - 


\ 

* . -. 


I 

. 


— 


To-(2y 


Hypotheses 






3 


'! 2 . 05 


j 0.89 


0.261 




2 (330- 50) HYP MOR SUPRT THY 

3 (262- 58) HYP IS A "HUNCH" 

7 (262-115) HYP-FROM IMAGINATN 


.64 
.70 
-71 


, .65 
! .68 

: .58 

i • 




5 

; . 






_12-(2) 


Models 


■ 


1 

1 

1 


3 


2,08 


0„91 


0.396 




1 (340- 31) MODELS-PIC OF ATOM 
5 (340- 35) MDL-SCALD UP PICT 
8 (340- 81) MDLS R DEFECTIVE 


.60 
.67 
.81 


! 

i .74 
' .71 
; .56 

1 


j 


1 

i 




i 


13- (2) 


Laws 






'3 


'1.94"" 


' 0.81 


0.126 


1 
1 


1 (350- 13) LAW-DSCB OBSRVTNS 

2 (350- 14) LAW IS PERMANENT 
6 (350-49) LAW-NTR MUST DO 


.79 .55 
.41 .66 

.74 ; .59 

1 




i 

1 

I 
} 


i 





ERIC 



Bates iZ^/2i) 



'^I-3-9-Mar-74 



The results of the factor analysis are reported in Table 9, 
Items with factor loading greater than 0.4 have been circled. 

Insert Table 9 about here 



Five of the seven factors can be readily interpreted; 

Factor I - Nature-Understandable 

II - Scientific Literalist 

III - Measurement-Approximate 
IV - 
V 

VI - Science Tentative 

VII - Scientific Methods 



^ Mixtures of misconceptions 



Items which loaded greater than 0.4 on only a single factor 
were grouped into new subscales which were examined by item 
analysis. The selected items have been marked with an asterisk 
in Table 9. The results are reported in Table 10. 

Insert Table 10 about here 



These f actor-subscales are encouraging. The homogeneous 
scales such as I (Nature-Understandable), lib (Literalist), 
and III (Measurement-Approximate) have reliabilities around 
0.4-0.5 for only 3-4 items. These can be expanded to strong 
scales by adding 6-7 similar items. The Factor-subscales 
VI (Science-Tentative) and VII (Scientific Methods) are more 
heterogeneous and would require 15-20 additional items to 
produce sufficiently reliable scales. Factor-subscale II 
(Literalist) requires further study with the addition of 
ERJC other examples of literal interpretation. 



! 4.) 



I 5 0'^30'*^^^'^vfo0004-r<^^P-^.MOt^nooO 

• e oooooooooooooooooooo 



oooooooJooaooooooo o o o o o 



a 



'> o o 
-5 

• • • 

I 



^ ^ It — 

'vj ^^ CO n/Hl-^ o — « 
*t ^ nJ- -</r^l^ o 
O ^i V rnj '-nlo rn 
• • • • • 

o o o o oVq^ o o o o 
I 



-« 4? o 
f\J ro 

O —4 r-4 



^ o 
1 



3 r\ O 'NJ >t Art O 

<^ CT* 1^ in r*- 'Ni^io 

o o o o n o\cyo 

II ^1 




^- iTN ^-^ — I CO o/^o\>t >t c^ o, 

• • • • t] • • • •) • 

"5 o a r>\nl o o o o 
II 11^ 111 




oooooooooooooooo 
II f I I I I I I « 



^ r*. >t 
N- 

o O o 



it) 



3 o o :3 o 



• • • • 

o o a o 



o o 



cr> st 
O r*- ng 

O -4 O 



o o o o 
f I 



f> in 

^^ ^ ^ ;n 

0 r> o ^ 
• • • • 

O -5 O O 

1 ( I 



O sO !7> 

4- in ^ \j ^ -\j 

o o o o o 

• « t • • t 

o o o o o o 

I I 




insOtNrMco-Tfno 
'V or — * in ^ t if\ 
o o o »^ rn "3 n 
• •••••tt 

^ -;:> o o o o "3 o 
I I I I 




O O 



o o o o o o o\ 
I • f 



O o -I o 'X) O n o CO 0 ^ ^ -< n o -o o o O .-^j ^ ^ 

o D — « ^ -I .Nj ^ ^ ^ ^ J ^ ^ ^ ,^ 

• •••••••••••••••#•••,##,•«,# 

"J -5 D c3 o ::5 :5 ^ o o 'J o o :> -> o o 3 o o o o o 
< < • I I I I I I f I t t 




^ fj^ >t tn 00 

O O O in 

a ^ o o o 

oooooooo 
It f 



o 

JO 




:5 -> o 




^ — 4 in 

in 'Nj m 

fNJ 

• • • • • 

o o o o o 





































n 


^+ 














a 










\I 








n 
















o 




in 




























O 


— ♦ 








"5 










* 


• 




t 


• 






• 


• 


• 


« 


• 


• 


t 




• 


• 








t-J 












-o 








O 








1 










1 


I 


1 


1 




1 




1 


1 




1 


1 


r 






<^ 




•.n 


-0 


-) 


•-^ 








J» 


in 


•J- 












{'1 




*j 






-4 






o 


>t- 




J" 
















~j 












—4 


-"^ 










« 


t 


• 


t 


• 


t 


• 


• 


• 


t 


c 


• 


t 


• 


• 


• 


• 




'3 








' J 






■ J 


o 






U 


(J 









I \ 



•n rg o O ai rg .j- 
rc J\ CD >r <\j ^ o sj 

I I I 











> 








qI 




o 






in 






• 


• 











in 


o 




o 








^1 




!^ 


a* 


t 


nj 










nl 




\^ 




o 










• 


• 1 


• 


\ • 


• 


• 


• 


• 


• 


« 1 








o 


o 




o 


'J 






1 








I 


• 




i 


• 





'J ! 

I ! 



to f 
•H : 

ro i 

>< ■ 

C 1 
< ! 



tl3 

k3 



o! 



*) 

./I .X 

a. 1 ^, 

> U 

3 ^ ->j 

*^ I- CrT 

;^ 'X 



^ r) uj V) 

^ x: '<j^ 'j:: <i 

«t «T i^") . ^ I ^ 

H- . J 

ij I -J 

O Jj 

.z .ii > — :d 

. I.- T 

o a > 

r: ^ -0 ^ 

O- K 

X o n -> 

-U ^ «I 4J 



u 

to >• 

■■•J 'O 

r. 

t a. 
« ^ 
I ^ 



I 

U I 



\ ^ ^ ^ 

< O O (J 

'/I ryi CO 



a 

I 



0 o t: n h-. ly. x i.t -j t s k 

iL . J ^ H- i.' :d x u x <r 

a h- 00 X o. V- :r tM. ^ 

x ju Q_ -J f*^ "3- ri c.^ 

X u. -J r: .r ^- c\. x 

J ..J -J i: X ::r r> t > 

I "? <wJ .J > »-« _j LO --I uo ' ) i/) 

> fV ^ ..J LU 4. 

1 ^ 1— i "-^ rg 0 :i* u. 

^ - > K i- - I I 

.1 -J .J X r. t/o ^ a 1 ci n. 

-) o X n X M un rr^ > >. >, > 

< «y) 'j ,u o u ^ >: 3 "."i x -L x r 



<l i-( 



O 



O u-> </? 



'JJ 



t3L _| «J 

I 



11. h- 

n uj 

> < 
< X 
3C UJ 



c:^ 



.'i »— a, 

> ;^ 

»-i > .jj 

o 1-0 i:) 

OJ 2i >. 

UL o a* 

■u i.iJ ;r 
^ 

(X CO l/i ^ 

O 

I i 

.-J j: -X 

n <i 

2: ^ -J J 



-t ^ 

o o 

rg :N 



r-- o ct) :) ^ o o 



a o o 



ERIC 



:^ O o o o o 

•f >t ^ J> <0 <NJ 

-4 -I ^ ^ 4. n 



•0 

a ^3 
•n ro 



^ -4 ^ -5 "5 ^ n in c?> cf' n 

•3" O Ni -^-^ rn u'> cr> 7> rsi ^ jS -n -< 

^ ^ ^ ^ ^ ^ 

o m in .J -3 O O o :i rg ^J o cm ^vi 

^ '^i o o gr: •t >^ rg rg -1 n ^ 



n 



\^ in 



00 ^ ^ rg 



U • 
D 

to 

O 

e 

to 



Bates {ZZ/ZD SPI-3-0-Mar-74 



TABLE 10 — Selected Items from SPI Factor Analysis 



1 

1 — f 


Factor 


Dif 


Dis 


# 


Mean 




KR#20 




Natur e— Under St andable 






■J 


X • 0*T 


1 m 

X « ux 


n c; 
U . D J J 




(120- 74) MIND UNDRST NTUR 
(120- 75) NATUR NVR UNDRSTD 
(120- 77) PBMS 2 CMPLX 2 EX 


.75 
.43 

o66 


.64 
.80 
.72 










II 


Literalist 






C 

D 




1 • U / 


U . 2o7 


i 
I 

I 
j 

i 
i 


(340- 31) MODELS-PIC OF ATM 
(340- 35) MDL-SCALD UP PICT 
(340- 36) MDLS MAY B MODIF 
(430- 41) SCI STS&END W FCT 
(265-130) ONLY 1 SOLN 2 TDM 


.60 
.67 
.87 
.49 

o89 


.62 
.64 
.50 
.36 
.38 














•5 
«j 


9 OR 


n Q 1 


n 

u . J 


(340- 31) MODELS-PIC OF ATM 
(340- 35) MDL-SCALD UP PICT 
(340- 81) MDLS R DEFECTIVE 


.60 
.67 
.81 


.74 

o71 
o56 










III 


Mp ^ ^iiT" ^^mp nl- — Ay^y^ytjV "i ttj^ "h p 








0 "5 Q 


n Q ^ 
U o o o 


U o 0 /o 


! 


(310- 91) 12 IN. IS APPROX 
(220-98) 15 IN-EXACT TRUTH 
(220-113) MSUR WTH N ERROR 


.67 
.79 
.93 


.85 
.82 
.50 












Science Tentative 






4 


2o95 


0.99 
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Conclusion 

Although we have failed to find usable subscales in the 
Science Process Inventory , this effort has been most encou jing 
for the future developrnent of a multidimensional instrum3nt to 
measure student *s knowledge about the nature and processes of 
science. There are also several guidelines for developing such 
an instrument • 

1 ) Theoretical Model A theoretical model which identifies 
the important dimensions of kr^wing about Llie nature and processes 
of science must be developed. A model will provide the basis 

for both interpreting the scales and for selecting items. This 
study suggests that the dimensions will be subtle and that 
considerable theoretical and empirical effort will be needed to 
identify them. 

2) Refinement of Scales The scales need to be quite well 
developed separately before techniques such as factor analysis 
will prove useful for "purifying*' the total instrument, 

3) Item Selection Items should meet criteria of difficulty 
and discriminating power similar to those suggested in the last 
section, i.e. mean difficulty ^ 0.5, discrimination > 0.4, 

4) Response Format The " agree"-"disagree" format provides 
little latitude for student response. If the purpose of the 
instrument is to describe rather than evaluate student's knowledge 
about the nature and processes of science, then a four point 

^ -ale (AA A D DD) with a confidence scale (L M H) may provide 
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more useful information. Alternate response formats surely 
need to be tested and evaluated during the development of 
a multidimensional instrument. 

5) Test Series The fact that a large portion of the SPI 
items are very easy for high school students suggests that they 
already have an acceptable understanding of many concepts about 
the nature and processes of science. These less difficult 
items might be included in a simpler instrument appropriate 

for upper grade school and early junior high school. 

6) Usefulness An instrument which is too cumbersome, 
esoteric, or long will not be of much interest to any but a 
few researchers. Although the amount of class time that can 
be allocated to testing is severely limited, it is reasonable 
to devote one class period to a regular testing program if the 
yield is rich enough. Thus, a multidimensional instrument should 
include some scales relating to the social aspects of science 
and an indication of student attitudes toward science. 

This paper has demonstrated that the development of a multi- 
dimensional instrument to measure student knowledge about the 
nature and processes of science is feasible. What is needed 
at this time is a cooperative effort on the pert of several 
science education researchers to advance the state of the art. 
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WELCH SCIENCE PROCESS INVENTORY 
Form D (Revised 1966) 



Directions 

The following statements are concerned with the activities, assumptions, pro- 
ducts, and ethics of science. Read each statement carefrlly and then mark your 
answer on the answer sheet provided. BLACKEN SPACE ONE vl) on the answer sheet 
if you generally AGREE with the statement; BLACKEN SPACE TWO (2) if yor gener- 
ally DISAGREE. 



Example : A D 

1. New York is a city. 1. 1 1 1 1 1 

2. Chicago is a mountain. 2. 1. B ^ 4 ^ 



Do not mark spaces 3, 4, or 5. If you change your mind, erase the first mark 
completely. Make no stray marks; they may count against you. 

Use a pencil to mark your answer sheet. DO NOT USE A PEN. Please supply in- 
formation concerning name, school, birth date, etc., in the appropriate spaces 
on the answer sheet. Do not write in the test booklet. 

Answer all statements. You will have 40 minutes which should give you suf- 
ficient time to finish all 135 statements. A scoring key has been established 
for the Inventory. Slightly more than half of the statements are keyed agree 
while the remainder are keyed disagree. 

Note carefully the numbering sequence on the answer sheet. 



DO NOT TURN THE PAGE UNTIL TOLD TO DO SO. 
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Blacken space 1 on your answer sheet if you generally AGREE with the statements- 
blacken space 2 if you generally DISAGREE. 

Dis- 
Agree agre e 

1. Surprising or unexpected observations have played an important 1 2 
role in the advance of science . 

2. The work of a scientist includes keeping a record of observa- 1 2 
tions. 

3. Scientists have differences of opinion about scientific matters. 1 2 

4. Careful observation is less important in modern science since 1 2 
the development of new instruments like the electron microscope, 

5. The observations a person makes are influenced by his past ex- 1 2 
perience . 



6. A scientist should make his findings available to the scien- 1 
tific community for independent confirmation, 

7. Theories are usually so well established, they do not require 1 
experimental testing. 

8. The essential test of a scientific theory is its use in pre- 1 
dieting future events. 

9. If two different hypotheses fit the observed facts, the -^xiiip- 1 
ler is accepted. 

10. An essential characteristic of the scientist is the ability to 1 
ask the right questions. 



11. If a researcher accurately reports his experimental procedures, 1 2 
other researchers will accept the experimental conclusions 

without question. 

12. Scientists assume there is order in the universe. 1 2 

13. A law of nature, such as Ohm's Law, is a statement that de- 1 2 
scribes what has been observed. 

14. Although a scientific hypothesis may have to be changed on the 1 2 
basis of new data, a physical law is permanent. 

15. A scientist wishes to make prejudiced observations of nature. 1 2 



16. 


Scientists should be unwilling to share their findings with 


1 


2 




other scientists. 






17. 


Assumptions in science are based on past experience. 


1 


2 


18. 


Theories suggest new relationships among facts. 


1 


2 


19. 


Science is a series of successively closer approximations to 


1 


2 




the truth. 






20. 


A scientist is often interested in finding relationships of 


1 


2 




the type, "when A occurs, then B will also occur." 






21. 


Scientists write articles for professional journals describing 


1 


2 




their research . 






22. 


Scientists do not make assumptions. 


1 


2 


23. 


Nature is not permitted to disobey the laws of science . 


1 


2 


24. 


Once a statement becomes a law of science, it will not be 


1 


2 




changed . 






25. 


A theory in science may be modified in light of new evidence. 


1 


2 
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26. 


An experiment is a set of conditions under which observations 


1 


2 




are made. 






27. 


Assumptions are not accepted until they are proven true. 


1 


2 


28. 


The knowledge of science is final. 


1 


2 


29. 


Scientists usually make observations of nature to answer spe- 


1 


2 




cific questions. 






30. 


Experimentation is principally concerned with proving the laws 


1 


2 




of nature . 







(Statements 31 - 36 refer to the following information.) The Bohr model of the 
atom is a description of the atom similar in form to the solar system. It has 
a central nucleus of protons and neutrons surrounded by electron orbits. State- 
ments 31 - 36 are concerned with this model. 



31. 


The model pictures the atom as we actually know it to exist. 


1 


2 


32. 


The model is a convenient way of representing the atom to help 


1 


2 




us understand it. 






33. 


The model presents an effective way of showing the different 


1 


2 




colors of the atomic particles. 






34. 


Scientific models are man-made. 


1 


2 


35. 


The model is a scaled-up picture of what scientists have seen 


1 


2 




in their microscopes. 






36. 


The model of the atom may be modified. 


1 


2 


37. 


Scientists do not make errors in their conclusions if they 


1 


2 




act "scientifically." 






38. 


A characteristic of scientific research is the use of instru- 


1 


2 




ments as aids to the senses. 






39. 


Experiments are used to test hypotheses. 


1 


2 


40. 


"Sea water contains salt," is an example of a scientific 


1 


2 




hypothesis . 






41. 


Science must start with facts and end with facts no matter what 


1 


2 




theoretical structures it builds in between. 






42. 


An accurate description of a scientific observation is a waste 


1 


2 




of time . 






43. 


A control in an experiment is used to give a check on factors 


1 


2 




not involved in the specific problem being studied. 






44. 


The assumptions in science are based on opinion, not fact. 


1 


2 


45. 


A law of nature is a description of what actually takes 


1 


2 




place, not a prescription of what must happen. 






46. 


It is the task of science to form theories to explain observa- 


1 


2 




tions . 






47. 


Two people looking at the same sunrise may see different 


1 


2 




things . 






48. 


If the results of an experiment do not agree with the previous 


1 


2 




answer, then the experiment is wrong. 






49. 


A law in science describes what nature must do. 


1 


2 


50. 


Hypotheses have more experimental support than theories. 


1 


2 
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Dis- 
Agree agree 

51. The main object of basic scientific research is the discovery 1 2 
of understanding rather than its practical application. 

52. A classification scheme, such as the periodic table of the 1 2 
elements, is based on common factors and differences noted 

in observations. 

53. Theories in science are often expressed as mathematical re- 1 2 
lationships. 

54. "We are going to have 36 snowfalls this winter" is an example 1 2 
of a scientific theory. 

55. The published results of scientists should be accepted with- 1 2 
out question. 

56. Collecting rocks is an example of scientific investigation. 1 2 

57. The point of an experiment is to set up a situation in which 1 2 
the control of variables is greater than it is in the ordin- 
ary course of events. 

58. A hypothesis is a simple guess or "hunch" that tries to ex- 1 2 
plain several observations. 

59. Scientific models are exact duplications of reality. 1 2 

60. Scientific conclusions should be based on facts, not opinion. 1 2 

61. Classification schemes are inherent in the materials classi- 1 2 
fied, rather than imposed on nature by the scientist. 

62. Induction is the process of generalising the characteristics 1 2 
of a class from observations of all of its members. 

63. Scientists view events today as clues to events in the past. 1 2 

64. The majority of newly suggested theories are accepted by the 1 2 
scientific community. 

65. Investigation of the possibilities of creating life in the 1 2 
laboratory is an invasion of science into areas where it 

doesn*t belong. 

(Items 66 - 77 are related to the following statement.) Those people who carry 
on the practice of science assume that: 

66. some mysterious occurrences do not have causes. 1 2 

67. all effects in nature have causes. 1 2 

68. if events A and B occur at the same time, then one must be 12 
the cause of the other. 

69. matter is an idea, not reality. 1 2 

70. events in nature are the result of discoverable causes. 1 2 

71. if a change in factor A leads to a change in factor B, then 1 2 
factor A is a cause of factor B. 

72. time can be measured. 1 2 

73. space does not exist. 1 2 

74. the human mind is capable of understanding the events and 1 2 
materials of nature. 

75. some natural things will never be understood. 1 2 

76. time is not real. 1 2 

77. some problems are too complex ever to be explained. 1 2 

78. Classification schemes are a useful means of organizing 1 2 
observations . 

79. Theories and hypotheses are often the result of comparisons. 1 2 

80. A hypothesis may be wrong. 1 2 
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81. All models used in science are somewhat dei'ective, 

82. If a scientist fails to solve a problem, it is probably 
because he did not follow the "scientific method." 

83. Grouping observations is an important part of scientific 
work. 

84. The scientist knows that his experiment will be successful 
if he follows the steps of the scientific method. 

85. A scientific hypothesis is essentially the same thing as a 
scientific fact. 



Dis- 
agree 

2 
2 

2 

2 

2 



86. One of the aims of science is to work towards more complex 1 2 
knowledge . 

87. A scientist should be skeptical of anything but his own 1 2 
work . 

88. "It's hot in this room," is a more precise observation than 1 2 
"It's 84 degrees Fahrenheit in this room." 

89. Deduction is the process of predicting particular occurrences 1 2 
from the general case. 

90. A scientist should attempt to disprove his own hypotheses. 1 2 



91. A measurement expressed as 12 inches is a statement of 12 
approximation . 

92. Scientists usually rely on outside authority for their con- 1 2 
elusions . 

93. The use of measurement is more evident in the biological 1 2 
sciences than the physical sciences. 

94. Prediction is an important goal of scientific investigation, 1 2 

95. The formulation of a theory is a means of explaining facts. 1 2 



96. 


If a choice is to be made between two theories , the more 


1 


2 




complex is chosen. 






97. 


To question the accuracy of Newton's theory of gravity would 


1 


2 




be unscientific. 






98. 


A measurement of length expressed as 15 inches is a state- 


1 


2 




ment of exact truth. 






99. 


"All swans are white. Penelope is a swan. Therefore, Pene- 


1 


2 




lope is white," is an example of deductive reasoning. 






100. 


"All matter consists of molecules," is an example of a 


1 


2 




scientific theory. 






101. 


A scientist prefers simple explanations of phenomena. 


1 


2 


102. 


Some presently accepted theories were opposed by other 


1 


2 




scientists when first proposed. 






103. 


Since a measurement involves the use of numbers, it cannot 


1 


2 




be wrong. 






104. 


Scientists assume a real world exists outside of the mind. 


1 


2 


105. 


A value of a hypothesis is its suggestion of new experiments. 


1 


2 



106. Scientific knowledge is in the process of development. 1 2 

107. Scientific investigations must follow definite approved 1 2 
procedures . 

108. A thermometer is an example of a measuring device. 1 2 

109. Scientists assume nature is likely to change suddenly. 1 2 

110 . A theory with ten supporting and two denying experiments 1 2 
is more likely to be accepted than a theory with four 

supporting and no denying experiments. 
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111. 


The law of conservation of energy is an example of an un- 


1 


2 




changing truth. 






112* 


Some scientific discoveries are the result of "luck." 


1 


2 


113. 


Physics is an exact science because physicists are able to 


1 


2 




make measurements without error. 






114. 


When a scientist makes a prediction, he is assuming that 


1 


2 




nature is consistent. 






115. 


Hypotheses may arise from imagination. 


1 


2 


116. 


The statements of science represent the best approximations 


1 


2 




available at the time. 






117. 


The basic principle of science is that discoveries and re- 


1 


2 




search should have practical application . 






118. 


Knowledge expressed in terms of numbers indicates a lesser 


1 


2 




degree of understanding than that knowledge which is not ex- 








pressed numerically . 






119. 


A scientist believes that an experimen t performed today wi 11 


1 


2 




produce the same results as the same experiment performed 








last week. 






120. 


Scientific knowledge is tentative . 


1 


2 


121. 


There is only one scientific method used by scientists. 


1 


2 


122. 


When confronted with a new problem, a scientist searches 


1 


2 




the literature to see what similar work has been done. 






123. 


A scientist assumes the same cause produces the same 


1 


2 




effect under the same conditions. 






124 . 


Applying the scientific method to a problem will always 


1 


2 




produce the correct answer. 






125 . 


Science is essentially statistical in nature and deals 


1 


2 




in terms of probabilities. 






126. 


Scientists assume a force due to gravitation is present on 


1 


2 




all bodies of the universe . 






127. 


Scientists believe occurrences in nature are predictable . 


1 


2 


128 . 


Scientists use "trial and error" approaches to problems 


1 


2 




with success • 






129. 


The primary objective of science is to develop new and- 


1 


2 




improved living conveniences. 






130 . 


Scientist A used one procedure to solve problem X, and 


1 


2 




scientist B used a di f f erent procedure to solve problem X . 








Both scientists solved the problem* This is impossible. 






131. 


A scientist is more likely to accept a theory on the basis 


1 


2 




of his personal ideas than on the experimental evidence 








available . 






132. 


It is important to express the "degree of estimate" in the 


1 


2 




findings of science. 






133. 


A scientist may be looking for the answer to one problem 


1 


2 




and find the answer to another. 






134. 


Experiments should be repeated, if possible. 


1 


2 


135. 


There are many methods of solving scientific problems. 


1 


2 
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Scoring Key 



1-A 


36-A 


71-A 


10$-A 


2-A 


37-D 


72-A 


107-D 


3-A 


38-A 


73-D 


108- A 


4-D 


39-A 


74-A 


109-D 


5-A 


AO-D 


75-D 


110-D 


6-A 


41-A 


76-D 


111-D 


7-D 


42-D 


77-D 


112-A 


8-A 


4 3-A 


7 8-A 


11 3-D 


9-A 


44-D 


79-A 


114-A 


10- A 


45-A 


80-A 


11 5-A 


11-D 


4 6-A 


81-A 


116-A 


12-A 


47-A 


82-D 


117-D 


13-A 


48-D 


83-A 


118-D 


14-D 


49-D 


84-D 


11 9-A 


15-D 


50-D 


85-D 


120-A 


16-D 


51-A 


86- D 


121-D 


17-A 


52-A 


87-D 


122-A 


18-A 


53-A 


88-D 


123-A 


19-A 


54-D 


89-A 


124-D 


20-A 


55-D 


90-A 


125-A 


21-A 


56-D 


91-A 


126-A 


22-D 


57-A 


92-D 


127-A 


23-D 


58-A 


93-D 


128- A 


24-D 


59-D 


94-A 


129-D 


25-A 


60- A 


95-A 


130-D 


26-A 


61-D 


96-D 


131-D 


27-D 


62-D 


97-D 


132-A 


28-D 


63-A 


98-D 


133-A 


29-A 


64-D 


99-A 


134-A 


30-D 


65-D 


100- A 


13 5-A 


31-D 


66-D 


101-A 




32-A 


67-A 


102-A 




33- D 


68-D 


103-D 




34-A 


69-D 


104- A 




35-D 


70- A 


105-A 
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APPENDIX B 
The 13 Hypothesized SPI Subscales 



NOTE: The correlations in these tables 
is the point biserial correlation. 
The column titled "Bates'* is the corre- 
lation between the item and the subscale. 
The column titled "Welch" is the corre- 
lation between the item and the total 
score on SPI (form B). 
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APPENDIX A 

Science Process Invent o ry 
Form D (Revised 1966) 



Wayne W. Welch 
University of Wisconsin 



SCIENCE PROCESS INVENTORY 
Scoring Key 



1-A . 


36-A 


71-A 


lOS-A 


2-A 


37-D 


72-A 


107-D 


3-A 


38-A 


73-D 


lOS-A 


4-D 


39-A 


74-A 


109-D 


5-A 


40-D 


75-D 


110-D 


6-A 


41-A 


76-D 


111-D 


7-D 


42-D 


7 7-D 


112-A 


8-A 


43-A 


78-A 


113-D 


9-A 


44-D 


79-A 


114-A 


10-A 


45-A 


80-A 


115-A 


11-D 


4 6-A 


81-A 


116-A 


12-A 


47-A 


82-D 


117-D 


13-A 


48-D 


83-A 


118-D 


14-D 


49-D 


84-D 


119- A 


15-D 


50-D 


85- D 


120-A 


16-D 


51-A 


86-D 


121-D 


17-A 


52-A 


87-D 


122-A 


18-A 


53-A 


88-D 


123-A 


19-A 


54-D 


89-A 


124-D 


20-A 


55-D 


90-A 


125-A 


21-A 


56-D 


91-A 


126-A 


22-D 


57-A 


92-D 


127-A 


23-D 


58-A 


93-D 


128-A 


24-D 


59-D 


94-A 


129-D 


25-A 


60- A 


95-A 


130-D 


26-A 


61-D 


96- D 


131-D 


27-D 


62-D 


97-D 


132-A 


28-D 


63-A 


98- D 


133-A 


29-A 


64-D 


99-A 


134-A 


30-D 


65-D 


100- A 


135-A 


31- D 


66-D 


101-A 




3 2-A 


67-A 


10 2-A 




33- D 


68-D 


103-D 




34-A 


69-D 


104-A 




35-D 


70-A 


105-A 





• ERIC 



APPENDIX B 
The 13 Hypothesized SPI Subscales 



NOTE: The correlations in these tables 
is the point biserial correlation. 
The column titled "Bates" is the corre- 
lation between the item and the subscale 
The column titled "Welch" is the corre- 
lation between the item and the total 
score on SPI (form B). 
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